FIG. 1 depicts a conventional video and/or audio (audio/video) playback device 10, such as a set top box. A set top box may receive audio/video from a remote location, from a closer location for example within the house of the user, and/or from storage within the set top box. For example, a set top box may receive files from remote locations such as a satellite, from a network, or from a cable service provider. A set top box might receive files from another storage device such as a computer in proximity to the set top box. The set top box may also play files from media such as a Blu-ray disk, or from internal storage. Thus, examples of a playback device may include devices such as a TiVo® box, high definition cable box, or Blu-ray player. The playback device 10 includes a main processor 12, additional processors 14, and is connected to a display 20. The additional processors 14 may include an audio-video signal demultiplexer, an audio decoder, a video decoder and/or other processors. The main processor 12 controls operation of the conventional playback device 10. The additional processors 14 are used to perform functions such as processing a video file. The IR/other inputs allow the playback device 10 to receive input, such as video files from other sources. The conventional playback device 10 outputs video to the display/speakers 20.
In order to play a video file or perform other operations, the main processor 12 calls the additional processors to perform specific operations. For example, in response to a request from a user to play a video file, the main processor 12 may communicate with the additional processors 14 to determine the amount of memory the audio and video decoders require, set aside a particular amount of memory, set the particular type of encoding used for the video being played, set various other properties for the audio and/or video decoders, and otherwise communicate with the additional processors. During playback of the video file, the user may request other operations such as turning on closed captioning, changing the volume or channel, and/or performing trick operations such as a pause, rewind, or fast forward. To perform these operations, the main processor 12 communicates with the additional processors 14. In addition, data from the video file, which is typically sent from the main processor 12, may continue to be received and provided to the additional processors 14 for processing and display using display/speakers 20. Thus, the main processor 12 continues to call and/or receive returns from the additional processors 14 throughout playback. Thus, through the operation of the main processor 12 and additional processors 14, operation of the conventional playback device 10 may continue.
Manufacturers of and/or purchasers of components for playback devices often desire to test the quality of their systems before providing them to end users. For example, they may desire to test whether software applications they have written work with a set top box and whether a video file from a network can be acceptably played on the set top box. In order to do so, the conventional playback device 10 is operated. Typically, testers play video files on the conventional playback device 10 and look for glitches appearing on the display 20 or heard on the speakers 20. The tester may utilize debug logs generated by the software application being tested. In addition, high level commands such as a user inputting a play command or other trick operation may be logged. Based on the results of such tests, the performance of the conventional playback device 10 may be evaluated.
Although conventional methods for testing the conventional playback device 10 exist, there are drawbacks. Most particularly, many conventional methods for testing the conventional playback device 10 perturb the performance of the conventional playback device 10. As a result, such methods may be unable to accurately assess performance of the conventional playback device 10. Use of debugging logs, for example, may result in a large number of debugging messages sent by the maker of the application. Storage of the debugging messages in human readable format, such as text strings, may consume a significant amount of memory and CPU resources. Such storage may also distort the video being played. Viewing played video files on the display and checking for glitches may indicate whether a particular file can be played. However, not all glitches are detectable by the human eyes and/or ears. In addition, viewing/hearing the glitches on the display 20 or speakers 20 may not indicate what is causing the glitches or how the issue might be solved. Similarly, logging of high level commands may not provide a fine enough view of operation of the playback device to adequately diagnose issues with performance. Thus, an improved method for evaluating the performance of a playback device is desired.